Skip to content

Performance & Benchmarks#149

Closed
WhiteBlackGoose wants to merge 9 commits intoLanguageDev:masterfrom
WhiteBlackGoose:perf
Closed

Performance & Benchmarks#149
WhiteBlackGoose wants to merge 9 commits intoLanguageDev:masterfrom
WhiteBlackGoose:perf

Conversation

@WhiteBlackGoose
Copy link
Contributor

@WhiteBlackGoose WhiteBlackGoose commented Jun 14, 2022 •
edited
Loading

Problem

Yoakke, unfortunately, turns out to have significant performance issues (compared to Antlr, for example). They are:

  1. Inefficient ParseResult and related types. Making it efficient may boost it to 10x, but it's still not enough
  2. A lot of nested interfaces. Every call on token stream is 5+ nested calls.

However, the parsing and lexing algos look to be fine. At least, I didn't get past 2., so cannot say for sure.

@WhiteBlackGoose
Copy link
Contributor Author

WhiteBlackGoose commented Jun 14, 2022

Initial results:

Method Mean Error StdDev
Parse 49.920 us 1.1544 us 3.2936 us
Lex 3.713 us 0.0708 us 0.1494 us

WhiteBlackGoose
WhiteBlackGoose commented Jun 14, 2022
View reviewed changes
Sources/SynKit/Benchmarks/Parser.Benchmarks/Parser.Benchmarks.csproj Outdated
Comment on lines 11 to 15
<ProjectReference Include="$(SolutionDir)\SynKit\Libraries\Parser\Parser.csproj" />
<ProjectReference Include="$(SolutionDir)\SynKit\Libraries\Lexer.Generator\Lexer.Generator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="False" />
<ProjectReference Include="$(SolutionDir)\SynKit\Libraries\Parser.Generator\Parser.Generator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="False" />
<PackageReference Include="BenchmarkDotNet" Version="0.13.1" />
<PackageReference Include="Yoakke.Synkit.Lexer" Version="2022.6.14-8.42.51-nightly" />
<PackageReference Include="Yoakke.Synkit.Lexer.Generator" Version="2022.6.14-8.42.51-nightly" />
<PackageReference Include="Yoakke.Synkit.Parser" Version="2022.6.14-8.42.51-nightly" />
<PackageReference Include="Yoakke.Synkit.Parser.Generator" Version="2022.6.14-8.42.51-nightly" />
Copy link
Contributor Author

@WhiteBlackGoose WhiteBlackGoose Jun 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BDN otherwise fails to run

@WhiteBlackGoose
Copy link
Contributor Author

WhiteBlackGoose commented Jun 14, 2022

image
Parser - [2022-06-14 20-26-04].zip

@WhiteBlackGoose
Copy link
Contributor Author

WhiteBlackGoose commented Jun 14, 2022

Replaced ParseError's binary OR with return first:

Method Mean Error StdDev
Parse 15.655 us 0.3817 us 1.0641 us
Lex 2.825 us 0.0562 us 0.0998 us

@WhiteBlackGoose
Copy link
Contributor Author

WhiteBlackGoose commented Jun 14, 2022

Removed creating a dictionary on ParseError

Method Mean Error StdDev
Parse 7.425 us 0.1222 us 0.1143 us
Lex 2.752 us 0.0516 us 0.0552 us

@WhiteBlackGoose
Copy link
Contributor Author

WhiteBlackGoose commented Jun 14, 2022

image
Parser - [2022-06-14 23-12-46].zip

@WhiteBlackGoose WhiteBlackGoose mentioned this pull request Jun 14, 2022
Draft
@WhiteBlackGoose
Copy link
Contributor Author

WhiteBlackGoose commented Jun 15, 2022 •
edited
Loading

Warning: this is wrong result, because it dysfunctions

Replaced public static ParseResult<T> operator |(ParseResult<T> first, ParseResult<T> second) with return first.IsOk ? first.Ok : first.Error;

Method Mean Error StdDev
Parse 919.3 ns 18.34 ns 25.71 ns
Lex 3,779.2 ns 73.81 ns 121.27 ns

But it looks wrong, so gotta make it functionally correct

@WhiteBlackGoose
Copy link
Contributor Author

WhiteBlackGoose commented Jun 15, 2022

For AngouriMath these optimizations have these impacts

Method Mean Error StdDev Median
ParseEasy OOB 11.61 ms 0.440 ms 1.298 ms 11.10 ms
ParseEasy Optim 1.143 ms 0.0510 ms 0.1371 ms 1.131 ms
ParseEasy Antlr 0.039 ms
ParseHard OOB 155.05 ms 9.570 ms 28.217 ms 146.27 ms
ParseHard Optim 69.306 ms 2.8647 ms 7.8420 ms 66.660 ms
ParseHard Antlr 4.4 ms

@WhiteBlackGoose
Copy link
Contributor Author

WhiteBlackGoose commented Jun 15, 2022

Warning: no merge! The Result type is intentionally broken!

@kant2002
Copy link
Collaborator

kant2002 commented Jan 1, 2024

@WhiteBlackGoose i'm new maintainer of the Yoakke, so if you still (after such long time) want to continue exploration of Yoakke performance, I'm here for you.

@kant2002
Copy link
Collaborator

kant2002 commented Mar 29, 2025

Dear @WhiteBlackGoose thanks for pushing question of the Yoakke performance. I create separate benchmark project in #173 so if you want, we can continue from that place. You are always welcome in this project. I will close this PR, but I think main findings are rightly point on the places which should be improved.

@kant2002 kant2002 closed this Mar 29, 2025
@WhiteBlackGoose
Copy link
Contributor Author

WhiteBlackGoose commented Mar 29, 2025

hi @kant2002 , sorry, missed your message, congrats on becoming a new maintainer of the project! Yeah I think it's currently not on my radar to contribute here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@WhiteBlackGoose @kant2002